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UTILIZING DATA REDUCTION IN STEGANOGRAPHIC 
AND CRYPTOGRAPHIC SYSTEMS 

SPECIFICATION 

5 FIELD OF THE INVENTION 

This invention relates to digital signal processing, and more particularly to a 
method and a system for encoding at least one digital watermark into a signal as a means of 
□ conveying information relating to the signal and also protecting against unauthorized 

m manipulation or use of the signal. 

SI 10 CROSS-REFERENCE TO RELATED APPLICATION 

■J3 This application is a continuation-in-part of PCT application No. PCT/US00/06522, 

™ . filed 1 4 March 2000 entitled "Utilizing Data Reduction in Steganographic and Cryptographic 

P Systems" (which claimed priority to U.S. Provisional Application No. 60/125,990, filed 24 

m March 1 999). This application also claims the benefit of the following applications: pending 

PJ 15 U.S. Patent Application Serial No. 09/046,627, filed 3/24/98, entitled "Method for Combining 
Transfer Function with Predetermined Key Creation"; pending U.S. Patent Application Serial 
No. 09/053,628, filed 04/02/98, entitled "Multiple Transform Utilization and Application for 
Secure Digital Watermarking"; pending U.S. Patent Application Serial No. 60/169,274, filed 
1 2/7/99, entitled "Utilizing Data Reduction in Steganographic and Cryptographic Systems"; 
20 and pending U.S. Patent Application Serial No. 60/147,134, filed 08/04/99, entitled, "A 
Secure Personal Content Server." All of the patent applications previously identified in this 
paragraph are hereby incorporated by reference, in thesir entireties. 



BACKGROUND OF THE INVENTION 

Many methods and protocols are known for transmitting data in digital form 
25 for multimedia applications (including computer applications delivered over public networks 
such as the internet or World Wide Web ("WWW"). These methods may include protocols 



for compression of data, such that it may more readily and quickly be delivered over limited 
bandwidth data lines. Among standard protocols for data compression of digital files may be 
mentioned the MPEG compression standards for audio and video digital compression, 
promulgated by the Moving Picture Experts Group. Numerous standard reference works and 
patents discuss such compression and transmission standards for digitized information. 

Digital watermarks help to authenticate the content of digitized multimedia 
information, and can also discourage piracy. Because piracy is clearly a disincentive to the 
digital distribution of copyrighted content, establishment of responsibility for copies and 
derivative copies of such works is invaluable. In considering the various forms of multimedia 
content, whether "master," stereo, NTSC video, audio tape or compact disc, tolerance of 
quality will vary with individuals and affect the underlying commercial and aesthetic value of 
the content. It is desirable to link copyrights, ownership rights, purchaser information or 
some combination of these and related data into the content in such a manner that the content 
must undergo damage, and therefore reduction of its value, with subsequent, unauthorized 
distribution, commercial or otherwise. Digital watermarks address many of these concerns. 
A general discussion of digital watermarking as it has been applied in the art may be found 
in U.S. Patent No. 5,687,236 (whose specification is incorporated in whole herein by 
reference). 

Such prior art applications have been drawn to providing basic digital 
watermarking functionality. For instance, it has been known to provide an apparatus or 
method for encoding or decoding independent information, including a digital watermark, 
represented as a series of data bits into or out of a series of digitized samples, wherein the 
apparatus contained: 

a) a sample buffer for holding, accessing, and transforming digitized samples; 

b) a digital signal processor for performing sample modifications and spectral 
transformations; 

c) a memory for storing information representing: 

1) a mask set, including one or more masks, 



2) a start of message delimiter (wherein at least one of the masks in question, 
or the start of message delimiter, are random or pseudo-random), 

3) a mask calculation buffer, 

4) a first buffer holding the independent information, 

5) an information bit index, 

6) a message size, representing an amount of information, 

7) one index into each of said one or more masks, 

8) a state of a decoding process, 

9) a table representing a map function, 

10) a flag indicating whether a complete message has been decoded or 
encoded, 

1 1) a number of samples for reading into said sample buffer, and 

12) a flag indicating a size of a message that has been decoded; 

d) a first input for acquiring a plurality of digital samples; 

e) a first output for outputting a plurality of modified digital samples; 

f) a second input for inputting a plurality of values to the one or more masks, the start 
of message delimiter, the mask calculation buffer, the first buffer, the table and the 
number of samples; 

g) a third output for outputting the independent information stored in the first buffer 
as a result of the decoding process and a value of the state of the decoding process 
to an attached digital circuit; 

h) one or more data buses for transferring information from: 

1 ) the first input to the sample buffer, 

2) the sample buffer to the digital signal processor, 

3) the digital signal processor to the sample buffer, 

4) the sample buffer to the first output, 

5) the second input to the memory, and 

6) the memory to the third output; and 



i) a clock for generating a clock signal for driving the digital signal processor and the 
data bus(es), and for controlling the operation of the apparatus. 

Further applications of basic digital watermarking functionality have also been 
developed. Examples of such applications are shown in U.S. Patent No. 5,889,868 (whose 
specification is incorporated in whole herein by reference). Such applications have been 
drawn, for instance, to implementations of digital watermarks that were deemed most suited 
to particular transmissions, or particular distribution and storage mediums, given the nature 
of digitally sampled audio, video, and other multimedia works. There have also been 
developed techniques for adapting watermark application parameters to the individual 
characteristics of a given digital sample stream, and for implementation of digital watermarks 
that are feature-based — i.e., a system in which watermark information is not carried in 
individual samples, but is carried in the relationships between multiple samples, such as in a 
waveform shape. For instance, natural extensions may be added to digital watermarks that 
may also separate frequencies (color or audio), channels in 3D while utilizing discreteness in 
feature-based encoding only known to those with pseudo-random keys (i.e., cryptographic 
keys) or possibly tools to access such information, which may one day exist on a quantum 
level. 

A matter of general weakness in digital watermark technology relates directly 
to the manner of implementation of the watermark. Many approaches to digital watermarking 
leave detection and decode control with the implementing party of the digital watermark, not 
the creator of the work to be protected. This weakness removes proper economic incentives 
for improvement of the technology. One specific form of exploitation mostly regards efforts 
to obscure subsequent watermark detection. Others regard successful over encoding using 
the same watermarking process at a subsequent time. Yet another way to perform secure 
digital watermark implementation is through "key-based" approaches. 

This paper draws a distinction between a "forensic watermark," based on 
provably-secure methods, and a "copy control" or "universal" watermark which is intended 
to be low cost and easily implemented into any general computing or consumer electronic 
device. A watermark can be forensic if it can identify the source of the data from which a 
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copy was made. For example, assume that digital data are stored on a disk and provided to 
"Company A" (the "A disk"). Company A makes an unauthorized copy and delivers the copy 
to "Company B" (the "B disk"). A forensic watermark, if present in the digital data stored on 
the "A disk," would identify the "B disk" as having been copied from the "A disk." 
5 On the other hand, a copy control or universal watermark is an embedded 

signal which is governed by a "key" which may be changed (a "session key") to increase 
security, or one that is easily accessible to devices that may offer less than strict cryptographic 
security. The "universal" nature of the watermark is the computationally inexpensive means 
_ for accessing or other associating the watermark with operations that can include playback, 

a — i 

•n 10 recording or manipulations of the media in which it is embedded. 

"I A fundamental difference is that the universality of a copy control mechanism, 

=F which must be redundant enough to survive many signal manipulations to eliminate most 

U casual piracy, is at odds with the far greater problem of establishing responsibility for a given 

_ instance of a suspected copying of a copyrighted media work. The more dedicated pirates 

2 1 5 must be dealt with by encouraging third party authentication with "forensic watermarks" or 
U those that constitute "transactional watermarks" (which are encoded in a given copy of said 

content to be watermarked as per the given transaction). 

3 The goal of a digital watermark system is to insert a given information signal 
or signals in such a manner as to leave little or no evidence of the presence of the information 

20 signal in the underlying content signal. A separate but equal goal is maximizing the digital 
watermark's encoding level and "location sensitivity" in the underlying content signal such 
that the watermark cannot be removed without damage to the content signal. 

One means of implementing a digital watermark is to use key-based 
security. A predetermined or random key can be generated as a map to access the hidden 

25 information signal. A key pair may also be used. With a typical key pair, a party possesses 
a public and a private key. The private key is maintained in confidence by the owner of 
the key, while the owner's public key is disseminated to those persons in the public with 
whom the owner would regularly communicate. Messages being communicated, for 
example by the owner to another, are encrypted with the private key and can only be read 



by another person who possesses the corresponding public key. Similarly, a message 
encrypted with the person's public key can only be decrypted with the corresponding 
private key. Of course, the keys or key pairs may be processed in separate software or 
hardware devices handling the watermarked data. 

Two conventional techniques for providing key-based confidentiality 
and/or authentication currently in use involve reciprocal and non-reciprocal encrypting. 
Both systems use non-secret algorithms to provide encryption and decryption, and keys 
that are used by the algorithm. 

In reciprocal algorithm systems, such as DES, the same key and algorithm 
is used both to encrypt and decrypt a message. To assure confidentiality and authenticity, 
the key should be known only to the sending and receiving computers, and were 
traditionally provided to the systems by "secure" communication, such as courier. 

In the prior art there have been developed systems wherein a common key 
may be developed by the sender and receiver using non-secure communications. In such 
systems, as described in U.S. Patents 4,200,770, 5,375,169 and 5,583,939, each party to 
a communication generates a numerical sequence, operates on the sequence and transfers 
the result to the other party. By further operation using the transferred result and the 
locally generated sequence, each party can develop the identical encyphering key, which 
cannot be obtained from the transferred results alone. 

As implemented for use over the internet, the most common prior art 
encryption systems are those denoted by the Secure Socket Layer (SSL) and IPSEC 
protocols. 

In non-reciprocal systems, such as described in U.S. Patent 4,218,582, a 
first party to a communication generates a numerical sequence and uses that sequence to 
generate non-reciprocal and different encrypting and decrypting keys. The encrypting key 
is then transferred to a second party in a non-secure communication. The second party 
uses the encrypting key (called a public key because it is no longer secure) to encrypt a 
message that can only be de-crypted by the decrypting key retained by the first party. The 
key generation algorithm is arranged such that the decrypting key cannot be derived from 



the public encrypting key. Similar methods are known for using non-reciprocal keys for 
authentication of a transmission. In this application, the non-secure "public" key is used to 
a message that has been encrypted using a secure "private" key known only to the 
originating party. In this method the receiving party has assurance that the origination of 
the message is the party who has supplied the "public" decrypting key. Prior art systems 
for key generation have often relied upon supposedly-random or quasi-random numbers 
generated by a fixed mathematical algorithm. 

Adaptations of key systems specifically used in conjunction with digital 
watermarking have been developed, as disclosed in, for example, U.S. Patent No, 
5,822,432 (which is incorporated in whole herein by reference). Such adaptations may 
include, for instance, providing methods for the human-assisted generation and application 
of pseudo-random keys for the purpose of encoding and decoding digital watermarks to 
and from a digitized data stream. In such methods, a pseudo-random key and key 
application "envelope" may be generated and stored using guideline parameters input by a 
human engineer interacting with a graphical representation of the digitized data stream. 
Key "envelope" information may be permanently associated with the pseudo-random 
binary string comprising the key. Key and "envelope" information may then be applied in 
a digital watermark system to the encoding and decoding of digital watermarks. Such a 
method may improve encoding and decoding with digital watermarks by providing: 
separation of the encoder from the decoder; increased information capacity (relative to 
spread spectrum methods); destruction or degradation of content when attempts to erase 
watermarks take place; detection of presence of watermarks without ability to access 
watermark information; multi-channel watermark capability; use of various classes of keys 
for watermark access control; support for alternative encoding, decoding, or other 
component algorithms; and/or use of a digital notary to authenticate and time stamp 
watermark certificates. 

While, as described above, various prior art approaches do exist for 
implementation of digital watermarking (though not necessarily for forensic or copy 
control use), there are additional desirable features for digital watermarking systems that 
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are not currently believed to be available. For instance, it would be desirable to be able to 
secure a data signal by using data reduction techniques to reduce the data signal into a 
reduced data signal; in conjunction with cryptographic techniques, so that an output signal 
can reliably and efficiently be securely delivered. 

It would further be advantageous to user remainder signals (produced by 
data reduction techniques) as a vehicle for performing encryption upon and using in 
conjunction with encrypting/decrypting of a data signal to be secured. 

It would likewise be desirable to combine data reduction techniques to 
reduce a data signal into a reduced data signal; produce a remainder signal from the data 
signal; and then embed complementary watermarks in reduced data signal and the 
remainder signal, for effective and secure delivery of an output signal. 

It would still further be desirable to combine scrambling techniques in 
conjunction with data reduction techniques such that data signals can be reduced and 
transmitted on a secured basis. 

It would likewise be desirable to provide cost-efficient and universal 
systems for digital watermarking, and to provide systems adaptable both to copy 
protection and forensic tracing of "pirated" data signals to detect and deter unauthorized 
copyists thereof. 

It would also be desirable to provide a system of digital watermarking that 
is highly compatible with known and future methods for compression of data used in 
conjunction with electronic transmission thereof 

It would further be desirable to provide digital watermarking techniques in 
conjunction with known and effective "key" systems for cryptography and data signal 
protection. 

The prior art does not meet these needs. 
SUMMARY OF THE INVENTION 

The present invention provides a method of securing a data signal which 
comprises the steps of: applying a data reduction technique to reduce the data signal into a 
reduced data signal; subtracting said reduced data signal from the data signal to produce a 
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remainder signal; embedding a first watermark into said reduced data signal to produce a 
watermarked, reduced data signal; and adding said watermarked, reduced data signal to said 
remainder signal to produce an output signal. 

The present invention also provides a method of securing a data signal which 
comprises the steps of: applying a data reduction technique to reduce the data signal into a 
reduced data signal; subtracting said reduced data signal from the data signal to produce a 
remainder signal; embedding a first watermark into said reduced data signal to produce a 
watermarked, reduced data signal; embedding a second watermark into said remainder signal 
to produce a watermarked remainder signal; and adding said watermarked, reduced data 
signal to said watermarked remainder signal to produce an output signal. 

The present invention also provides a method of securing a data signal which 
comprises the steps of: applying a data reduction technique to reduce the data signal into a 
reduced data signal; subtracting said reduced data signal from the data signal to produce a 
remainder signal; using a first scrambling technique to scramble said reduced data signal to 
produce a scrambled, reduced data signal; using a second scrambling technique to scramble 
said remainder signal to produce a scrambled remainder signal; and adding said scrambled, 
reduced data signal to said scrambled remainder signal to produce an output signal. 

The present invention also provides a method of securing a data signal which 
comprises the steps of: applying a data reduction technique to reduce the data signal into a 
reduced data signal; subtracting said reduced data signal from the data signal to produce a 
remainder signal; using a first cryptographic technique to encrypt the reduced data signal to 
produce an encrypted, reduced data signal; using a second cryptographic technique to encrypt 
the remainder signal to produce an encrypted remainder signal; and adding said encrypted, 
reduced data signal to said encrypted remainder signal to produce an output signal. 

The present invention also supplies a system for securing a data signal which 
comprises: means to apply a data reduction technique to reduce the data signal into a reduced 
data signal; means to subtract said reduced data signal from the data signal to produce a 
remainder signal; means to apply a first cryptographic technique to encrypt the reduced data 
signal to produce an encrypted, reduced data signal; means to apply a second cryptographic 
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technique to encrypt the remainder signal to produce an encrypted remainder signal; and 
means to add said encrypted, reduced data signal to said encrypted remainder signal to 
produce an output signal. 

The present invention also supplies a system for securing a data signal which 
comprises: (a) a computer processor; (b) at least one computer memory; (c) a data reduction 
algorithm; and (d) at least one digital watermarking algorithm, wherein said computer 
processor is supplied with programming in conjunction with said computer memory: (I) to 
apply said data reduction algorithm to the data signal to yield a reduced data signal, and to 
subtract said reduced data signal from the data signal to produce a remainder signal; (II) 
to embed a first watermark into said reduced data signal by application of said at 
least one digital watermarking algorithm to produce a watermarked, reduced data signal; (III) 
to embed a second watermark into said remainder signal by application of said at least one 
digital watermarking algorithm to produce a watermarked remainder signal; and (IV) to add 
said watermarked, reduced data signal to said watermarked remainder signal to produce an 
output signal. 

The present invention also provides a method of securing a data signal which 
comprises the steps of: evaluating the data signal to determine its characteristics and 
reducibility; selecting at least one appropriate data reduction technique for the data signal 
based on the data signal's characteristics; applying said at least one appropriate data 
reduction technique to the data signal to produce a reduced data signal; embedding at least 
one digital watermark in the reduced data signal; and supplying an output signal 
corresponding to the data signal, said output signal comprising said watermark and said 
reduced data signal. 

The present invention also supplies a method for the protection of a data 
signal, comprising the steps of: (a) defining and analyzing a plurality of data substreams 
within the data signal; (b) associating at least one key or key pair with data reduction digital 
watermarking for at least one of said data substreams; (c) employing said at least one key or 
key pairs for at least one step selected from the group of: (i) identifying at least one associated 
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watermark; (ii) encoding at least one associated watermark; (iii) detecting at least one 
associated watermark; or (iv) decoding at least one associated watermark. 

A method for protected distribution of a data file is also provided, which 
method comprises: (a) embedding one or more digital watermarks in the data file using data 
reduction techniques in creating said digital watermark; (b) and distributing the digitally 
watermarked file to an end user. 

Also provided is a method for analyzing a data signal that has been embedded 
with at least one digital watermark using a data reduction technique, said method comprising: 
receiving the data signal; processing the data signal to detect information relative to the digital 
watermark; analyzing the detected information to determine if output of the data signal is 
authorized; and outputting said data signal if the detected information establishes that output 
is authorized. 

Also provided is a device for analyzing a data signal that has been embedded 
with at least one digital watermark using a data reduction technique, said device comprising: 
an interface for receiving the data signal; a detector for processing the data signal to detect 
information relative to the at least one digital watermark; an analyzer to analyze the detected 
information to determine if output of the data signal is authorized or unauthorized; and an 
signal generator to output data if the detected information establishes that output is 
authorized. 

There are two design goals in an overall digital watermarking system's low 
cost, and universality. Ideally, a method for encoding and decoding digital watermarks in 
digitized media for copy control purposes should be inexpensive and universal. This is 
essential in preventing casual piracy. On the other hand, a more secure form of protection, 
such as a "forensic watermarks," can afford to be computationally intensive to decode, but 
must be unaffected by repeated re-encoding of a copy control watermark. An ideal method 
for achieving these results would separate the signal into different areas, each of which can 
be accessed independently. The embedded signal or may simply be "watermark bits" or 
"executable binary code," depending on the application and type of security sought. 
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Improvements to separation have been made possible by enhancing more of the underlying 
design to meet a number of clearly problematic issues. 

The present invention interprets the signal as a stream which may be split into 
separate streams of digitized samples or may undergo data reduction (including both lossy and 
lossless compression, such as MPEG lossy compression and Meridian's lossless compression, 
down sampling, common to many studio operations, or any related data reduction process). 
The stream of data can be digital in nature, or may also be an analog waveform (such as an 
image, audio, video, or multimedia content). One example of digital data is executable binary 
code. When applied to computer code, the present invention allows for more efficient, 
secure, copyright protection when handling functionality and associations with predetermined 
keys and key pairs in software applications or the machine readable versions of such code in 
microchips and hardware devices. Text may also be a candidate for authentication or higher 
levels of security when coupled with secure key exchange or asymmetric key generation 
between parties. The subsets of the data stream combine meaningful and meaningless bits of 
data which may be mapped or transferred depending on the application intended by the 
implementing party. The present invention utilizes data reduction to allow better 
performance in watermarking as well as cryptographic methods concerning binary executable 
code, its machine readable form, text and other functionality-based or communication-related 
applications. Some differences may simply be in the structure of the key itself, a pseudo 
random or random number string or one which also includes additional security with special 
one way functions or signatures saved to the key. The key may also be made into key pairs, 
as is discussed in other disclosures and patents referenced herein. The present invention 
contemplates watermarks as a plurality of digitized sample streams, even if the digitized 
streams originate from the analog waveform itself The present invention also contemplates 
that the methods disclosed herein can be applied to non-digitized content. Universally, data 
reduction adheres to some means of "understanding"the reduction. This disclosure 
contemplates data reduction which may include down sampling, lossy compression, 
summarization or any means of data reduction as a novel means to speed up watermarking 
encode and decode operations. Many forms of data reduction rely upon sampling of a data 
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signal, for instance frequency or time sampling of a digital audio or video signal. For 
example, a signal may be sampled on a regular basis every x fractions of a second, where x 
is arbitrarily chosen, such that representative data slices of the signal are obtained. Other data 
reduction techniques include bit depth reduction. Bit depth reduction relies on the fact that 
when measuring items, scales of different degrees of precision can be used. For example, one 
can measure things on a scale with three division marks (zero to two), or on a scale of the 
same magnitude with ten division marks (zero to nine). Scales with more divisions are of 
higher precision than scales with fewer divisions. On a computer, because of processing and 
storage limitations, numerical values (e.g., numerical values relating to a digitized signal) are 
also represented with varying degrees of precision. For example, one can use two bits (a 
scale of zero to three) to represent a numerical value or use five bits (a scale of zero to 
thirty-one) to represent the same numerical value. The number of bits used to represent a 
numerical value is generally referred to as the "bit depth." Numerical data may be reduced 
for storage or transmission by reduction of the bit depth scale. 

While any of a number of different data reduction techniques can be used in 
conjunction with the present invention, essentially a lossy method for data reduction may yield 
the best results for encode and decode operations. Data reduction methods should be 
appropriately chosen with an eye toward the particular type of data signal being reduced. 
Some data signals may more readily be reduced than others. For instance, when the data 
reduction technique chosen is a compression technique, it will be realized that not all data 
signals or files are equally compressible. For example, there are limits to the degree to which 
aesthetic information (such as music or video signals) may be compressed without losing their 
aesthetic or informational value. Thus, in practicing the present invention, techniques can be 
applied for intelligent selection of data reduction, and differential data reduction techniques 
can be selected for differential substreams of an aggregate data stream. For example, a 
computer processor implementing the present invention for protection of a data signal stream 
comprising, say, both video and text portions, can be programmed to "split" the aggregate 
data stream into video and text signal substreams, and to apply a first data reduction 
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algorithm most suitable for video data to the first substream, while applying a second data 
reduction algorithm most suitable for text data to the second substream. 

It is desirable to have both copy control and forensic watermarks in the same 
signal to address the needs of the hardware, computer, and software industries while also 
providing for appropriate security to the owners of the copyrights. This will become clearer 
with further explanation of the sample embodiments discussed herein. 

The present invention also contemplates the use of data reduction for purposes 
of speedier and more tiered forms of security, including combinations of these methods with 
transfer function functions. In many applications, transfer functions (e.g., scrambling), rather 
than mapping functions (e.g., watermarking), are preferable or can be used in conjunction 
with mapping. With "scrambling," predetermined keys are associated with transfer functions 
instead of mapping functions, although those skilled in the art may recognize that a transfer 
function is simply a subset of mask sets encompassing mapping functions. It is possible that 
tiered scrambling with data reduction or combinations of tiered data reduction with 
watermarking and scrambling may indeed increase overall security to many applications. 

The use of data reduction can improve the security of both scrambling and 
watermarking applications. All data reduction methods include coefficients which affect the 
reduction process. For example, when a digital signal with a time or space component is 
down sampled, the coefficient would be the ratio of the new sample rate to the original 
sample rate. Any coefficients that are used in the data reduction can be randomized using the 
key, or key pair, making the system more resistant to analysis. Association to a 
predetermined key or key pair and additional measure of security may include biometric 
devices, tamper proofing of any device utilizing the invention, or other security measures. 

Tests have shown that the use of data reduction in connection with digital 
watermarking schemes significantly reduces the time required to decode the watermarks, 
permitting increases in operational efficiency. 

Particular implementations of the present invention, which have yielded 
extremely fast and inexpensive digital watermarking systems, will now be described. 
These systems may be easily adapted to consumer electronic devices, general purpose 
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computers, software and hardware. The exchange of predetermined keys or key pairs may 
facilitate a given level of security. Additionally, the complementary increase in security for 
those implementations where transfer functions are used to "scramble" data, is also 
disclosed. 

BRIEF DESCRIPTION OF THE FIGURES 

For a more complete understanding of the invention and some advantages 
thereof, reference is now made to the following descriptions taken in connection with the 
accompanying drawings in which: 

FIG. 1 is a functional block diagram that shows a signal processing system that 
generates "n" remainder signals and "n" data reduced signals. 

FIG. 2 is a functional block diagram for an embodiment of the present 
invention which illustrates the generation of an output signal comprised of a data-reduced, 
watermarked signal and a first remainder signal. 

FIG. 3 is a functional block diagram for an embodiment of the present 
invention which illustrates the generation of an output signal comprised of a data-reduced, 
watermarked signal and a watermarked, first remainder signal. 

FIG. 4 is a functional block diagram for decoding the output signal generated 
by the system illustrated in FIG. 2. 

FIG. 5 is a functional block diagram for decoding the output signal generated 
by the system illustrated in FIG. 3. 

FIG. 6 is a functional block diagram for an embodiment of the present 
invention which illustrates the generation of an output signal comprised of a data-reduced, 
scrambled signal and a first remainder signal. 

FIG. 7 is a functional block diagram for an embodiment of the present 
invention which illustrates the generation of an output signal comprised of a data-reduced, 
scrambled signal and a scrambled, first remainder signal. 

FIG. 8 is a functional block diagram for decoding the output signal generated 
by the system illustrated in FIG. 6. 

FIG. 9 is a functional block diagram for decoding the output signal 
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generated by the system illustrated in FIG. 7. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

The embodiments of the present invention and its advantages are best 
understood by referring to the drawings, like numerals being used for like and corresponding 
parts of the various drawings. 
An Overview 

A system for achieving multiple levels of data reduction is illustrated in FIG. 
1 . An input signal 1 0 (for example, instructional text, executable binary computer code, 
images, audio, video, multimedia or even virtual reality imaging) is subjected to a first data 
reduction technique 1 00 to generate a first data reduced signal 20. First data reduced signal 
20 is then subtracted from input signal 10 to generate a first remainder signal 30. 

First data reduced signal 20 is subjected to a second data reduction technique 
101 to generate a second data reduced signal 21. Second data reduced signal 21 is then 
subtracted from first data reduced signal 20 to generate a second remainder signal 3 1 . 

Each of the successive data reduced signals is, in turn, interactively subjected 
to data reduction techniques to generate a further data reduced signal, which, in turn, is 
subtracted from its respective parent signal to generate another remainder signal. This 
process is generically described as follows. An (n-1) data reduced signal 28 (i.e, a signal that 
has been data reduced n-1 times) is subjected to an nth data reduction technique 109 to 
generate an nth data reduced signal 29. The nth data reduced signal 29 is then subtracted 
from the (n-1) data reduced signal 28 to produce an n* remainder signal 39. 

An output signal can be generated from the system illustrated in FIG. 1 in 
numerous ways. For example, each of the n remainder signals (which, through represented 
by reference numerals 30-39, are not intended to be limited to 10 signals though n must 
obviously be a finite number, and as a practical matter will usually be comparatively small) 
and the n* data signal may optionally subjected to a watermarking technique, or even 
optionally subjected to a encryption technique, and each of the (n+1) signals (whether 
watermarked or encrypted, or otherwise untouched) may then be added together to form an 
output signal. By way of more particular examples, each of the (n+1) signals (i.e., the n 
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remainder signals and the n* data reduced signal) can be added together without any 
encryption or watermarking to form an output signal; or one or more of the (n+ 1 ) signals may 
be watermarked and then all (n+1) signals may be added together; or one or more of the 
(n+1) signals may be encrypted and then all (n+1) signals may be added together. It is 
anticipated that between these three extremes lie numerous hybrid combinations involving one 
or more encryptions and one or more watermarkings. 

Each level may be used to represent a particular data density. E.g., if the 
reduction method is down-sampling, for a DVD audio signal the first row would represent 
data sampled at 96 kHz, the second at 44.1 kHz., the third at 6 kHz., etc. There is only an 
issue of deciding what performance or security needs are contemplated when undertaking the 
data reduction process and choice of which types of keys or key pairs should be associated 
with the signal or data to be reduced. Further security can be increased by including block 
ciphers, special one way functions, one time stamps or even biometric devices in the software 
or hardware devices that can be embodied. Passwords or biometric data are able to assist in 
the determination of the identity of the user or owner of the data, or some relevant identifying 
information. 

A variety of keys may advantageously be chosen. Additionally, any key or 
keys employed need not remain static over time but may be changed from time to time. For 
instance, the key may be changed in real time, or upon detection of a "marker" signal within 
the data signal stream. The key can also be a ciphered key. As may be known in the art, the 
key or keys may be generated by any of a variety of effective methods, including 
steganographic cipher, symmetric cryptographic cipher, and asymmetric cryptographic cipher. 
Keys may be derived (in whole or in part) from the signal stream itself or may be derived from 
sources completely external to the signal stream. 

Additionally, and given that information signals may comprise a variety of 
forms of information (e.g., audio, still image, video, computer code, or text), it is appreciated 
that a single multimedia information signal stream may be divided into multiple substreams 
based on the various constituent information forms in the multimedia information stream. It 
maybe advantageous, in such a substreamed context, to associate predetermined discrete, and 
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particular, forms or instances of key or key pair to particular information substreams « for 
instance, a predetermined first key or key pair could be assigned for association and use with 
a video substream whereas a predetermined second key or key pair could be assigned for 
association and use with a text substream. Thus, complex watermarking of a multi-substream 
data signal may be flexibly accomplished. Such complexity may contribute, inter alia, to more 
effective watermarking and security as multiple watermarks would have to be compromised 
in order to compromise the entire aggregate information stream or set of substreams. Keys 
and key pairs are understood to be multifunctional, insofar as they are useful for both the 
encoding and decoding of watermarks. 

An example of a real world application is helpful here. Given the predominant 
concern, at present, of MPEG 1 Layer 3, or MP3, a perceptual lossy compression audio data 
format, which has contributed to a dramatic re-evaluation of the distribution of music, a 
digital watermark system must be able to handle casual and more dedicated piracy in a 
consistent manner. The present invention contemplates compatibility with MP3, as well as 
any perceptual coding technique that is technically similar. One issue, is to enable a universal 
copy control "key" detect a watermark as quickly as possible from a huge range of perceptual 
quality measures. For instance, DVD 24 bit 96 kHz, encoded watermarks, should be detected 
in at least "real time," even after the signal has been down sampled, to say 12 kHz of the 96 
kHz originally referenced. By delineating and starting with less data, since the data-reduced 
signal is obviously smaller though still related perceptually to the original DVD signal, 
dramatic increases in the speed and survival of the universal copy control bits can be achieved. 
The present invention also permits the ability to separate any other bits which may be 
associated with other more secure predetermined keys or key pairs. 

Where the data stream is executable computer code, the present invention 
contemplates breaking the code into objects or similar units of functionality and allowing for 
determination of what is functionally important. This may be more apparent to the developer 
or users of the software or related hardware device. Data reduction through the use of a 
subset of the functional objects related to the overall functionality of the software or 
executable code in hardware or microchips, increase the copyright protection or security 
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sought, based on reducing the overall data to be associated with predetermined keys or key 
pairs. Similarly, instead of mapping functions, transfer functions, so-called "scrambling," 
appear better candidates for this type of security although both mapping and transferring may 
be used in the same system. By layering the security, the associated keys and key pairs can 
be used to substantially improve the security and to offer easier methods for changing which 
functional "pieces" of executable computer code are associated with which predetermined 
keys. These keys may take the form of time- sensitive session keys, as with transactions or 
identification cards, or more sophisticated asymmetric public key pairs which may be changed 
periodically to ensure the security of the parties' private keys. These keys may also be 
associated with passwords or biometric applications to further increase the overall security 
of any potential implementation. 

An example for text message exchange is less sophisticated but, if it is a time 
sensitive event, e.g., a secure communication between two persons, benefits may also be 
encountered here. Security may also be sought in military communications. The ability to 
associate the securely exchanged keys or key pairs while performing data reduction to 
enhance the detection or decoding performance, while not compromising the level of security, 
is important. Though a steganographic approach to security, the present invention more 
particularly addresses the ability to have data reduction to increase speed, security, and 
performance of a given steganographic system. Additionally, data reduction affords a more 
layered approach when associating individual keys or key pairs with individual watermark 
bits, or digital signature bits, which may not be possible without reduction because of 
considerations of time or the payload of what can be carried by the overall data "covertext" 
being transmitted. 

Layering through data reduction offers many advantages to those who seek 
privacy and copyright protection. Serialization of the detection chips or software would 
allow for more secure and less "universal" keys, but the interests of the copyright owners are 
not always aligned with those of hardware or software providers. Similarly, privacy concerns 
limit the amount of watermarking that can be achieved for any given application. The 
addition of a pre-determined and cryptographic key-based "forensic" watermark, in software 
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or hardware, allows for 3rd party authentication and provides protection against more 
sophisticated attacks on the copy control bits. Creating a "key pair" from the "predetermined" 
key is also possible. 

Separation of the watermarks also relates to separate design goals. A copy 
control mechanism should ideally be inexpensive and easily implemented, for example, a form 
of "streamed watermark detection." Separating the watermark also may assist more 
consistent application in broadcast monitoring efforts which are time-sensitive and ideally 
optimized for quick detection of watermarks. In some methods, the structure of the key 
itself, in addition to the design of the "copy control" watermark, will allow for few false 
positive results when seeking to monitor radio, television, or other streamed broadcasts 
(including, for example, Internet) of copyrighted material. As well, inadvertent tampering 
with the embedded signal proposed by others in the field can be avoided more satisfactorily. 
Simply, a universal copy control watermark may be universal in consumer electronic and 
general computing software and hardware implementations, but less universal when the key 
structure is changed to assist in being able to log streaming, performance, or downloads, of 
copyrighted content. The embedded bits may actually be paired with keys in a decode device 
to assure accurate broadcast monitoring and tamper proofing, while not requiring a 
watermark to exceed the payload available in an inaudible embedding process. E.g., A full 
identification of the song, versus time-based digital signature bits, embedded into a broadcast 
signal, may not be recovered or may be easily over encoded without the use of block ciphers, 
special one way functions or one time pads, during the encoding process, prior to broadcast. 
Data reduction as herein disclosed makes this operation more efficient at higher speeds. 

A forensic watermark is not time sensitive, is file-based, and does not require 
the same speed demands as a streamed or broadcast-based detection mechanism for copy 
control use. Indeed, a forensic watermark detection process may require additional tools to 
aid in ensuring that the signal to be analyzed is in appropriate scale or size, ensuring signal 
characteristics and heuristic methods help in appropriate recovery of the digital watermark. 
Simply, all aspects of the underlying content signal should be considered in the embedding 
process because the watermarking process must take into account all such aspects, including 
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for example, any dimensional or size of the underlying content signal. The dimensions of the 
content signal may be saved with the key or key pair, without enabling reproduction of the 
unwatermarked signal. Heuristic methods may be used to ensure the signal is in proper 
dimensions for a thorough and accurate detection authentication and retrieval of the 
embedded watermark bits. Data reduction can assist in increasing operations of this nature 
as well, since the data reduction process may include information about the original signal, 
for example, signal characteristics, signal abstracts, differences between samples, signal 
patterns, and related work in restoring any given analog waveform. 

The present invention provides benefits, not only because of the key-based 
approach to the watermarking, but the vast increase in performance and security afforded the 
implementations of the present invention over the performance of other systems. 

The architecture of key and key-pair based watermarking is superior to 
statistical approaches for watermark detection because the first method meets an evidentiary 
level of quality and are mathematically provable. By incorporating a level of data reduction, 
key and key paired based watermarking is further improved. Such levels of security are 
plainly necessary if digital watermarks are expected to establish responsibility for copies of 
copyrighted works in evidentiary proceedings. More sophisticated measures of trust are 
necessary for use in areas which exceed the scope of copyright but are more factually based 
in legal proceedings. These areas may include text authentication or software protection 
(extending into the realm of securing microchip designs and compiled hardware as well) in 
the examples provided above and are not contemplated by any disclosure or work in the art. 

The present invention may be implemented with a variety of cryptographic 
protocols to increase both confidence and security in the underlying system. A predetermined 
key is described as a set of masks: a plurality of mask sets. These masks may include primary, 
convolution and message delimiters but may extend into additional domains. In previous 
disclosures, the functionality of these masks is defined solely for mapping. Public and private 
keys may be used as key pairs to further increase the unlikeliness that a key may be 
compromised. Examples of public key cryptosystems may be found in the following U.S. 
Patents Nos: 4,200,770; 4,218,582; 4,405,829; and 4,424,414, which examples are 
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incorporated herein by reference. Prior to encoding, the masks described above are generated 
by a cryptographically secure random generation process. Mask sets may be limited only by 
the number of dimensions and amount of error correction or concealment sought, as has been 
previously disclosed. 

A block cipher, such as DES, in combination with a sufficiently random seed 
value emulates a cryptographically secure random bit generator. These keys, or key pairs, 
will be saved along with information matching them to the sample stream in question in a 
database for use in subsequent detection or decode operation. These same cryptographic 
protocols may be combined with the embodiments of the present invention in administering 
streamed content that requires authorized keys to correctly display or play said streamed 
content in an unscrambled manner. As with digital watermarking, symmetric or asymmetric 
public key pairs may be used in a variety of implementations. Additionally, the need for 
certification authorities to maintain authentic key-pairs becomes a consideration for greater 
security beyond symmetric key implementations, where transmission security is a concern. 
Signal Processing in a Multi-watermark System (A Plurality of Streams May Be 
Watermarked) 

FIG. 2 illustrates a system and method of implementing a multiple- watermark 
system. An input signal 1 1 (e.g., binary executable code, instruction text, or other data), is 
first processed by a lossy data-reduction scheme 200 (e.g., down-sampling, bit-rate reduction, 
or compression method) to produced a data-reduced signal 40. Data-reduced signal 40 is 
then embedded with a watermark (process step 3 00) to generate a watermarked, data-reduced 
signal 50, while a copy of the unmarked, data-reduced signal 40 is saved. 

Watermarking process step 300 may be chosen from among various 
watermarking processes known in the art. As an example, a digital audio data signal may be 
represented, for purpose of watermarking, by a series of samples in 1 dimension. {S b S 2 , S 3 
. . S n }. This series is also referred to as a sample stream. The sample stream approximates 
an analog waveform of sound amplitude over time. Each sample represents an estimate of the 
wave amplitude at the instant of time the sample is recorded. For monaural audio, there is 
one such sample stream. Stereo audio is comprised of two sample streams, one representing 
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the right channel, and the second representing the left channel (or a combined left and right 
signal from which the left channel may be derived using the right channel). Each stream is 
used to drive a corresponding speaker to reproduce the stereo sound. What is referred to 
as CD quality audio is characterized by 16 bit (2 byte) stereo samples, recorded at 44. 1 Khz, 
5 or 44, 1 00 samples per second in each channel. The dynamic range of sound reproduction is 
directly proportional to the number of bits per sample. Some lower quality recordings are 
done at 8 bits. A CD audio recording can be stored using any scheme for containing the 2 
sample streams in their entirety. When these streams are played back at the same frequency 
they were recorded at, the sound recorded is reproduced to a high degree of accuracy. The 
I 1 0 sample stream is processed in order from first sample to last. For the purpose of the invention 
1 disclosed, the stream is separated into sample windows, each of which has a fixed number of 

consecutive samples from the stream, and where windows do not overlap in the sample 
stream. Windows may be contiguous in the sample stream. For illustration, assume each 
window contains 128 samples, and that windows are contiguous. Thus, the windows within 
1 5 the stream look like 

{ > S 1 , S 2 , S 3 . . . S 128 !, >S 12 9, S 130 , S ni . . . S 256 ! . . . >S n . 12 g. . . S n !} 
wherein the bracketed set > ... ! denotes each window and any odd samples at the end 
of the stream which do not completely fill a window can be ignored, and simply passed 
through the system unmodified. 
20 These windows will be used as input for the discrete Fast Fourier Transform 

(and its inverse) operation. Briefly, Fourier Transform methods are based on the principle 
that a complex waveform, expressed as amplitude over time and represented by a sample 
stream, is really the sum of a number of simple waveforms, each of which oscillates at 
different frequencies. By complex, it is meant that the value of the next sample is not easily 
25 predicted from the values of the last N samples or the time of the sample. By simple it is 
meant that the value of the sample is easily predictable from the values of the last N samples 
and/or the time of the sample. 

The sum of multiple simple waves is equivalent to the complex wave. The 
discrete FFT and its inverse simply translate a limited amount of data from one side of this 
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equivalence to the other, between the complex waveform and the sum of simple waves. The 
discrete FFT can be used to translate a series of samples representing amplitude over time (the 
complex wave, representing a digital audio recording) into the same number of samples 
representing total spectral energy in a given range of frequencies (the simple wave 
components) at a particular instant of time. This instant is the time in the middle of the 
original amplitude/time samples. The inverse discrete FFT translates the data in the other 
direction, producing the complex waveform, from its simpler parts. 

Each 128 sample window will be used as an input to the discrete FFT, 
resulting in 1 28 bins representing each of 1 28 frequency bands, ranging from 0 Hz to 22 Khz 
(the Nyquist frequency, or Vi the sampling rate). 

A watermark may be encoded into the audio signal in the frequency domain 
or in the time domain. In the latter case, no FFT or inverse FFT is necessary. However, 
encoding in the frequency domain is recommended, since its effects are scattered over the 
resultant time domain samples, and not easily predicted. In addition, frequency domain 
encoding makes it more likely that randomization will result in noticeable artifacts in the 
resultant signal, and therefore makes the stega-cipher more defensible against such attacks. 
It is in the frequency domain that additional information will be encoded into the audio signal 
for the purpose of this discussion. Each frequency band in a given time slice can potentially 
be used to store a small portion of some additional information to be added to the signal. 
Since these are discrete estimates, there is some room for error which will not significantly 
effect the perceived quality of the signal, reproduced after modification, by the inverse FFT 
operation. In effect, intentional changes, which cannot be distinguished from random 
variations, are introduced in the frequency domain, for the purpose of storing additional 
information in the sample stream. These changes are minimized so as not to adversely affect 
the perceived quality of the reproduced audio signal, after it has been encoded with additional 
information in the manner described below. In addition, the location of each of these changes 
is made virtually impossible to predict, an innovation which distinguishes this scheme from 
simple steganographic techniques. 
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The saved, unwatermarked data-reduced signal (signal 40) is subtracted from 
the original input signal 1 1 , yielding a remainder signal 60 composed only of the data that was 
lost during the data-reduction. A second watermark is then applied using a desired 
watermarking protocol (process step 301) to remainder signal 60 to generate a watermarked 
remainder signal 70. Finally, the watermarked remainder 70 and the watermarked, data- 
reduced signal 50 are added to form an output signal 80, which is the final, full-bandwidth, 
output signal. 

The two watermarking techniques (process steps 300 and 301) may be 
identical (i.e., be functionally the same), or they may be different. 

To decode the signal, a specific watermark is targeted. Duplicating the data- 
reduction processes that created the watermark in some cases can be used to recover the 
signal that was watermarked. Depending upon the data-reduction method, it may or may not 
be necessary to duplicate the data-reduction process in order to read a watermark embedded 
in a remainder signal. Because of the data-reduction, the decoding search can occur much 
faster than it would in a full-bandwidth signal. Detection speed of the remainder watermark 
remains the same as if there were no other watermark present. 

FIG. 4 illustrates a functional block diagram for one means of decoding the 
output signal generated by the system illustrated in FIG. 2. A signal to be analyzed 80 (e.g., 
the same output from FIG. 2) is processed by a data-reduction scheme 200. Data reduced 
signal 41 can then be decoded to remove the message that was watermarked in the original 
data reduced signal. Further, data reduced signal 41 can be subtracted from signal to be 
analyzed 80 to form a differential signal 61 which can then be decoded to remove the message 
that was watermarked in the original remainder signal. A decoder may only be able to 
perform one of the two decodings. Differential access and/or different keys may be necessary 
for each decoding. 

Additionally, the watermarking described in connection with this embodiment 
above may be done with a plurality of predetermined keys or key pairs associated with a 
single watermark "message bit," code object, or text. Keys or key pairs may also be stored 
or archived in a central certification authority, such that there will be a verified and official 
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version of a particular key or key pair whenever access to such key or key pair, or verification 
or identification of the legitimacy and authorization of the use of a particular data signal or 
file associated with that key, is required. The central certification authority could be, for 
instance, a secure computer server archive maintained by a copyright holder to store keys 
5 relating to copyrighted files watermarked using such keys. 



Signal Processing in a Single Watermark System 

FIG. 3 illustrates a system and method of implementing a single watermark 
system. The process and system contemplated here is identical to process described in 
connection to FIG. 2, above, except that no watermark is embedded in the remainder signal. 

^ 1 0 Hence, the watermarked, data-reduced signal 50 is added directly to the remainder signal 60 

jJj to generate an output signal 90. 

£ Additionally, the watermarking described in connection with this embodiment 

lI above may be done with a plurality of predetermined keys or key pairs associated with a 

J3 single watermark "message bit," code object, or text. 

□ 1 5 In either process, an external key can be used to control the insertion location 

m 

Tj, of either watermark. In a copy-control system, a key is not generally used, whereas in a 

jTj forensic system, a key must be used. The key can also control the parameters of the data- 

□ reduction scheme. The dual scheme can allow a combination of copy-control and forensic 
watermarks in the same signal. A significant feature is that the copy-control watermark can 

20 be read and rewritten without affecting the forensic mark or compromising its security. 

FIG. 5 illustrates a functional block diagram for one means of decoding the 
output signal generated by the system illustrated in FIG. 3. A signal to be analyzed 90 (e.g., 
the same output from FIG. 3) is processed by a data-reduction scheme 200. Data reduced 
signal 41 can then be decoded to remove the message that was watermarked in the original 

25 data reduced signal. 



Signal Processing in a Multi-scrambler System (A Plurality of Streams Mav Be Scrambled) 
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FIG. 6 illustrates a system and method of implementing a multi-scrambler 
system. An input signal 12 (e.g., binary executable code, instruction text, or other data), is 
first processed by a lossy data-reduction scheme 400 (e.g. , down-sampling, bit-rate reduction, 
or compression method) to produced a data-reduced signal 45. Data-reduced signal 45 is 
then scrambled using a first scrambling technique (process step 500) to generate a scrambled, 
data-reduced signal 55, while a copy of the unscrambled, data-reduced signal 45 is saved. 

The saved, unscrambled data-reduced signal (signal 45) is subtracted from the 
original input signal 1 2, yielding a remainder signal 65 composed only of the data that was 
lost during the data-reduction. A second scrambling technique is then applied (process step 
501) to remainder signal 65 to generate a scrambled remainder signal 75. Finally, the 
scrambled remainder signal 75 and the scrambled data-reduced signal 55 are added to form 
an output signal 85, which is the final, full-bandwidth, output signal. 

The two scrambling techniques (process steps 500 and 501) may be identical 
(i.e., be functionally the same), or they may be different. 

Additionally the scrambling described in connection with this embodiment 
may be done with a plurality of predetermined keys or key pairs associated with a single 
scrambling operation containing only a "message bit," code object, or text. 

To decode the signal, unscrambling follows the exact pattern of the scrambling 
process except that the inverse of the scrambling transfer function is applied to each portion 
of the data, thus returning it to its pre-scrambled state. 

FIG. 8 illustrates a functional block diagram for one means of decoding the 
output signal generated by the system illustrated in FIG. 6. A signal to be analyzed 85 (e.g., 
the same output from FIG. 6) is processed by a data-reduction scheme 200. Data reduced 
signal 46 can be subtracted from signal to be analyzed 85 to form a differential signal 66, 
which signal can then be descrambled in process 551 using the inverse transfer function of the 
process that scrambled the original remainder signal (e.g., the inverse of scrambling process 
501). Descrambling process 551 generates an descrambled signal 76. Data reduced signal 
46 may further be descrambled in process 550 using the inverse transfer function of the 
process that scrambled the original data reduced signal (e.g., the inverse of scrambling 
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process 500). Descrambling process 550 generates an descrambled signal 56, which may then 
be added to descrambled signal 76 to form an output signal 98. 

Signal Processing in a Single Scrambling Operation 

FIG. 7 illustrates a system and method of implementing a single scrambling 
system. The process and system contemplated here is identical to process described in 
connection to FIG. 6, above, except that no scrambling is applied to the remainder signal. 
Hence, the scrambled data-reduced signal 55 is added directly to the remainder signal 65 to 
generate an output signal 95. 

Additionally the scrambling described in connection with this embodiment 
may be done with a plurality of predetermined keys or key pairs associated with a single 
scrambling operation containing only a "message bit," code object, or text. 

FIG. 9 illustrates a functional block diagram for one means of decoding the 
output signal generated by the system illustrated in FIG. 7. A signal to be analyzed 95 (e.g., 
the same output from FIG. 7) is processed by a data-reduction scheme 200. Data reduced 
signal 46 can be subtracted from signal to be analyzed 95 to form a differential signal 66. 
Data reduced signal 46 may further be descrambled in process 550 using the inverse transfer 
function of the process that scrambled the original data reduced signal (e.g., the inverse of 
scrambling process 500). Descrambling process 550 generates an descrambled signal 56, 
which may then be added to differential signal 66 to form an output signal 99. Sample 
Embodiment: Combinations 

Another embodiment may combine both watermarking and scrambling with 
data reduction. Speed, performance and computing power may influence the selection of 
which techniques are to be used. Decisions between data reduction schemes ultimately must 
be measured against the types of keys or key pairs to use, the way any pseudo random or 
random number generation is done (chaotic, quantum or other means), and the amount of 
scrambling or watermarking that is necessary given the needs of the system. 

It is quite possible that some derived systems would yield a fairly large 
decision tree, but the present invention offers many benefits to applications in security that 
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are not disclosed in the art. 

As a further illustrative example of an advantageous embodiment, the 
following briefly describes an implementation of the present invention using sample rate 
reduction as the chosen data reduction method for watermarking in connection with an audio 
data signal. 

I. Encoding: 

- Audio data is downsampled from the original sample rate to 10 kHz. 

- The 10 kHz signal is upsampled to the original sample rate, yielding the 10 
kHz upsample. 

- The 10 kHz upsample is subtracted from the .original, yielding the 10 kHz 
upsample difference. 

- The 10 kHz signal is downsampled to 5 kHz. 

- The 5 kHz signal is upsampled to the 10 kHz, yielding the 5 kHz upsample. 

- The 5 kHz upsample is subtracted from the 10 kHz signal, yielding the 5 
kHz upsample difference. 

- The 5 kHz signal is marked with an open watermark (universal key for 
universal access), yielding the 5 kHz watermark. 

- The 5 kHz upsample difference is marked with a secure watermark (one key 
per encode), yielding the 10 kHz watermark. 

- The 5 kHz watermark is upsampled to 10 kHz, yielding the 5 kHz 
upsampled watermark. 

- The 5 kHz upsampled watermark is summed with the 1 0 kHz watermark, to 
yield the 10 kHz watermark sum. 

- The 10 kHz watermark sum is upsampled to the original sample rate, 
yielding the 10 kHz upsampled watermark. 

- The 10 kHz upsampled watermark is summed with the 10 kHz upsample 
difference to produce the output signal. 

11(A). Decoding both watermarks, or just the secure watermark: 

-Audio data is downsampled from the original sample rate to 10 kHz. 
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- The 10 kHz signal is downsampled to 5 kHz. 

- The 5 kHz signal is upsampled to the 1 0 kHz, yielding the 5 kHz upsample. 

- The 5 kHz upsample is subtracted from the 10 kHz signal, yielding the 5 
kHz upsample difference. 

- The open watermark is decoded from the 5 kHz. 

- The secure watermark is decoded from the 5 kHz upsample. 
IIB. Decoding just the open watermark: 

- Audio data is downsampled from the original sample rate to 5 kHz. 

- The open watermark is decoded from the 5 kHz. 

In connection with the above-described embodiment, alternative step IIB is 
illustrated because decoding the open watermark may have to occur on consumer electronic 
devices, and therefore, generally, fewer processing steps may be desirable in consumer 
electronic devices. The secure watermark is not as time-critical during the decode process, 
and can therefore be afforded more processing time. Note further that the original sample 
rate during the encode does not have to be the same as the original sample rate for decode. 
Any intervening sample rate conversion will be ignored, as long as it never drops below the 
same rate of the signal to which the watermark is applied (for example, 10 kHz for the 
secured watermark of the prior example, or 5 kHz for the open watermark of the prior 
example). 

The embodiments described herein may advantageously be implemented in 
connection with a data signal recipient's personal computer system or workstation 
(comprising a computer processor such as an Intel Pentium processor, spreadsheet software 
such as Microsoft Excel, and implementing a communications module such as a common web 
browser such as Internet Explorer or Netscape), linked by a World Wide Web connection to 
a data signal or file provider utilizing similar standard computer hardware and software, but 
may also be implemented in connection with any output device having appropriate electronic 
memory and/or processing capacity to implement the techniques set forth herein (which could 
include, for instance, consumer electronics output devices other than microcomputers). 
Because the digital watermarking techniques and systems disclosed herein are substantially 
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universal, however, they may be applied across a variety of computer hardware and software 
configurations, for use with a variety of transmitted data signals or files, over a variety of 
public or private networks (although the utility of the present invention for digital 
watermarking of audio or video files transmitted over public networks such as the internet is 
obvious). The network communication link between the data signal/file recipient and the 
signal/file provider may further be provided with some network-default level of encryption 
(perhaps a relatively weak level such as 56 bit encryption). Similarly, known computer 
programming techniques and languages (for instance, Visual Basic, JAVA, C++) may be 
adapted in a variety of fashions for use in either the data reduction steps discussed herein, the 
cryptographic/scrambling processes disclosed, the specific watermarking techniques applied, 
or any combination of the above, for customized data reduction and digital watermarking, and 
output of an output signal, in the fashion most amenable to a particular user's needs. The 
ability to adapt a wide range of data processing algorithms (including but not limited to 
algorithms for data reduction, encryption/decryption, and compression) to yield various 
desired data signal outputs, to apply customizable digital watermarking procedures, and to 
allow customizable and maximally-efficient forensic or copy control watermarks to popular 
and useful data transmission protocols, all across a broad range of computer system platforms 
(i.e., various hardware, software, computer language, and operating system combinations) 
provides the present invention with considerable versatility. 

The present invention as implemented with such computer systems permits 
secured delivery of valuable data streams over a variety of networks. Specifically, the present 
invention provides great utility for the delivery (commercial or otherwise) of video, audio, or 
other such files on media or over a public network such as the internet in a fashion that 
impedes theft or unauthorized use or resale of such files. For instance, the methods of the 
present invention could be applied to all the digitized commercial music files of a music 
vendor (to impose, for instance, a copy control watermark thereupon). Subsequently, those 
watermarked music files may be delivered to end users. End user attempts to make 
unauthorized copies can thus be controlled. Alternatively, output devices may be 
programmed to detect watermarks embedded in files by use of the present invention, such that 
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if the file does not contain an appropriate watermark, the output device will not execute or 
"play" the file. 

It is important to note that the watermarks embedded using the present 
invention may be embedded at a wide variety of points along the distribution chain for the 
data signals. For instance, in an embodiment in which the present invention is used to 
watermark commercial music or video files downloaded by an individual end user from a 
central server over an internet connection through an internet service provider, the present 
invention could be used to impose a forensic watermark (uniquely identifying the customer 
and download transaction) at the central server (or at the server of the internet service 
provider). When a suspected unauthorized copy of the file in question was detected, the 
watermark therein could be sensed/decoded in order to identify the source of the 
unauthorized copy. As has been emphasized, the techniques of the present invention may be 
applied to a wide variety of data signals, whether stored multimedia or computer code files, 
streamed files transmitted in real time, or other files or data signals, and may be applied in 
context-sensitive fashion to maximize protection (and effective signal transmission and 
output) for a particular data stream. It is also an aspect of this invention that the novel 
techniques for watermarking using data reduction herein can be exploited at the end user 
point of the distribution chain for data signals; that is, using the unique watermark/key 
information associated with a file watermarked using the techniques described hereinabove, 
a file may be analyzed (whether by representatives of a file copyright owner, for instance, or 
by hardware, software, or other appropriate analyzer, such as an embedded firmware chip, 
etc., contained in or supplied to an end user output device). Once the data signal is analyzed 
at the end user point, information relative to the any watermark or key actually contained on 
the file at that point may be derived and analyzed to determine if the file has been properly 
distributed to the end user. If it has not, the output device may be programmed to deny 
output or to manipulate the data signal in a destructive way (or to take other appropriate legal 
or copyright control actions as may be desired by the file owner). The present invention 
includes such uses of (and devices for) data reduction-derived watermark detection and 
output control. 
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Those of ordinary skill in the art will appreciate that the foregoing discussion 
of certain preferred embodiments is illustrative only, and does not limit the spirit and scope 
of the present invention, which are limited only by the claims set forth below. 



